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BACKGROUND OP THE INVENTION 

1. Field of the Invention 

The present invention relates to a parallel computer having 
a hierarchy structure, and more particularly, to a parallel 
computer that may be most applied to image processing that 
requires enormous amount of calculation , computer 
entertainments, and execution of scientific calculations. 

2. Description of the Related Art 

In conventional parallel computers, for instance, a 
conventional parallel computer having a common bus structure 
(or a common bus system) , a plurality of processors implemented 
with a plurality of semiconductor chips are arranged through 
a common bus formed on a semiconductor substrate* In this 
configuration, in order to further reduce the traffic of the 
common bus, a oaohe memory is incorporated In each layer when 
the common bus Is formed in a hierarchy structure. 

In general, a multiprocessing computer system includes 
two or more processors that execute computing tasks* In this 
system, other processors execute other computing tasks that are 
independent from the above-dedicated computing task while one 
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processor executes a de d icated computing task, or the 
multi-processing computer system divides a specified computing 
task Into plural execution elements , and then the plurality of 
processors In the multi-processing computer system execute these 
plural elements In order to reduce the total execution time of 
the computing task* In general, the processor la a device to 
execute operands of more than one and to generate and outputs 
the execution result. That ls # an arithmetic operation Is 
performed according to Instructions executed by the processor. 

A general structure of an available multi-processing 
computer system has a symmetry multiprocessor (SMP) structure. 
In a typical example, the multiprocessing computer system of 
the SMP structure Incorporates plural processors that are 
connected to a common bus through a cache hierarchy structure. 
In addition, a oonuura memory that is used for the processors 
In this system is also connected to the common bus. An access 
to a specified memory location In the common memory Is executed 
In a seme time during access to other memories. Because each 
memory location in the common memory is accessed uniformly, the 
structure of the oojmnon memory is called to as a uniform memory 
architecture (DMA) . 

In many oases, the processors and an internal cache are 
Incorporated in a computer system. In a SMP computer system, 
one or more cache memories are formed between a processor and 
a common bus in cache hierarchy. The computer system having the 
common bus structure operates based on a cache coherency In order 
to maintain the common memory model In which a specific address 
indicates a data Item preciously at any time. 

In general, when the result of arithmetic operation of 
data stored In a m em or y field corresponding to a specific memory 
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address lias been copied to a oaohe memory In a oaohe layer, the 
arithmetics operation la In a coherent state. For example, when 
a, data Item stored In a memory field addressed by a specific 
address is updated , tbe updated data Item will be copied to the 
oaohe memory that has stored a previous data Item . Or , the previous 
data item Is nullified In a stage and the updated data item is 
transferred from the main memory in a following stage. In the 
common bus system, a snooping bos protocol is commonly used* 
Bach coherent transaction that will be executed on the common 
bus is snooped (or detected) by comparison to the data item In 
the oaohe memory. When a copied data Item that Is affected by 
the execution of the above - coherent transaction is detected, 
the cache line belonging to the copied data item is updated 
according to the above-coherent transaction * 

The common bus structure, however, has several drawbacks 
to limit the feature of the multi-processing computer system. 
That is, there is a peak bandwidth (namely, the number of bytes 
per second to be transferred on the bus) to be used In the bus. 
When additional processors are connected to the common bus, tbe 
bandwidth for transferring data and instruction to the additional 
processors is over this peak bandwidth. When the bandwidth of 
one processor to be used is over the available bus bandwidth, 
some processors enter a waiting state until the bus bandwidth 
may be available. This reduces the performance of the computer 
system. In general, the maximum number of the processors to be 
connected to the common bus Is approximately 32. Plural 
processors are connected t:o the common bus, the capacity load 
of the bus is increased and the physical length of the common 
bus is also increased. When the capacity load and the length 
of the bus are Increased, the delay of the signal transfer on 



the bus is also Increased, The Increasing of the delay of the 
signal transfer also causes the Increasing of the execution time 
of a transaction* Accordingly, the plural processors are added 
Into the common bus, the peak bandwidth of the bos is also 
decreased. 

These drawbacks described above are more increased by 
increasing the performance of the processor and operation 
frequency. 

The micro-architecture of processors improved for a 
hlgh-f reqnency demand requires a higher bandwidth when compared 
with the bandwidth for processors in a previous generation even 
If a same number of processors is connected to a bus . Accordingly , 
the bus having the adequate bandwidth for a multiprocessing 
computer system in a previous generation can not satisfy the 
demand of a current computer system including processors of a 
high performance. Further , there is a drawback that it becomes 
difficult to make a programming model and to perform a debug 
the multi-processing systems otJier than the systems having the 
common bus structure. 

There is therefore a requirement to have an architecture 
of a new multi-processing system that is capable of executing 
processors in parallel even if the performance of a 
microprocessor and a peripheral circuit is increased and also 
even if the number of processors to be connected to a bus is 
increased. 

SUMMARY OF THE INVENTION 

Accordingly , an object: of the present Invention is, with 
due consideration to the drawbacks of the conventional technique „ 
to provide a parallel computer having a hierarchy structure 



capable of executing Id parallel high-speed processors of a 
desired number that have been made based on a leading edge 
technology. 

In accordance with a preferred embodiment of the present 
Invention, a parallel computer having a hierarchy structure 
ooniprlses an upper processing unit for executing a parallel 
processing task In parallel, and a plurality of lower processing 
units connected to the upper processing unit through a connection 
line - In the parallel computer , the upper processing unit divides 
the parallel processing task to a plurality of subtasks. and 
assigns the plurality of subtasks to the corresponding lower 
processing units and transfers data to be required for executing 
the plurality of subtasks to the lower processing units. The 
lower processing units execute the corresponding subtasks from 
the upper processing unit, and Inform the completion of the 
execution of the corresponding subtasks to the upper processing 
unit when the execution of the subtasks is completed, and the 
upper processing unit completes the parallel processing task 
when receiving the information of the completion of the execution 
from all of the lower processing units. 

In accordance with a preferred embodiment of the present 
Invention, a parallel computer having a hierarchy structure 
comprises an upper processing unit for executing a parallel 
processing task in parallel, a plurality of Intermediate 
processing units connected to the upper processing unit through 
a first connection line, and a plurality of lower processing 
units connected to the Intermediate processing units through 
a second connection line. In the parallel computer, the upper 
processing unit divides the parallel processing task to a 
plurality of first subtasks, and assigns the plurality of first 



subtasks to the corresponding Intermediate processing units, 
and transfers data to be required for executing the plurality 
of first subtasks to tbe Intermediate processing units. The 
Intermediate processing units divide tbe first subtasks to a 
S plurality o£ second subtasks , and assigns the plurality of second 
sub tasks to tbe corresponding lover processing units, and 
transfers data to be required for executing tbe plurality of 
second subtasks to the lower processing units. The lower 
processing units execute the corresponding second subtasks, and 
10 Inform the completion of the execution of the second subtasks 
□ to the corresponding intermediate processing units when the 

0 execution of all of the second subtasks is completed* The 

T[ intermediate processing units Inform the completion of the 

O execution of the corresponding second subtasks to the upper 

Ul 16 processing units when the execution of all of the first subtasks 
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is completed. The upper processing unit completes the parallel 
processing task when receiving the information of the completion 
of the execution from all of the Intermediate processing units. 
□ In the parallel oowpnter described above, the lower 

20 processing units conneoted to the connection line are mounted 
on a smaller area when compared with the upper processing unit r 
and a signal line through which each lower processing unit Is 
connected has a smaller wiring capacity , and an operation 
frequency for the lower processing units is higher than that 
25 for the upper- processing unit. 

In the parallel computer described above , the lower 
processing units connected to -the second connection line are 
mounted on a smaller area when compared with the Intermediate 
processing units connected to the first connection line, and 
30 a signal line through which each lower processing unit Is 
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connected lias a smaller wiring capacity, and an operation 
frequency for the lower processing units la M gh^r than that 
for the Intermediate processing units. 

In the parallel computer described above , each of the upper 
processing unit and the lower processing units has a processor 
and a memory connected to the processor* 

In the parallel computer described above , each of the upper 
processing unit, the Intermediate processing units, and the lower 
processing units has a processor and a memory connected to the 
processor. 

In the parallel computer described above, the upper 
processing unit received information regarding the completion 
of the subtasfc from each lower processing unit through a status 
signal line. 

In the parallel computer described above, each 
intermediate processing unit and the upper processing unit 
receives information regarding the completion of the second 
subtask and the first subtask from each lower processing unit 
and each intermediate processing unit through a status signal 
line, respectively. 

In the parallel computer described above, each lower 
processing unit comprises a processor, and a memory and a DMA 
controller connected to the processor. 

In the parallel counter described above , each 
intermediate processing unit comprises a processor, and a memory 
and a DMA controller connected to the processor. 

In the parallel computer described above, the processor 
and the DMA controller are connected In a coprocessor connection . 

In the parallel computer described above, the upper 
processing unit compresses the data to be required for executing 



the subtasks , and then transfers the compressed data to the 
corresponding lower processing unit a. 

In the parallel computer described above, the upper 
processing unit compressed the data to be required for executing 
the first sub tasks, and then transfers the compressed data to 
the corresponding Intermediate processing unite. 

In the parallel computer described above, each 
Intermediate processing unit compresses the data to be required 
for executing the second aubtaaka, and then transfers the 
compressed data to the corresponding lower processing units. 

In the parallel computer described above, each 
Intermediate processing unit Is a DMA transfer processing unit. 

In the parallel computer described above , the DMA. txansf er 
processing unit is a progran«nable. 

In the parallel computer- described above, each lower 
processing unit Is mounted with the upper processing unit as 
a multi-chip module on a board* 

In the parallel computer described above, each 
Intermediate processing unit and the corresponding lower 
processing units are mounted with the upper processing unit as 
a multi-chip module on a board. 

In the parallel computer described above, each of the 
upper processing unit and the lower processing units Is formed 
on an Independent semiconductor chip, and each semiconductor 
Qhlp is mounted as a single multi-chip module. 

In the parallel computer described above, each of the 
intermediate processing units , the corresponding lower 
processing units, and the upper processing unit is formed on 
an Independent semiconductor chip, and each semiconductor chip 
ia mounted as a single multi-chip module. 
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In the parallel computer described above, a structure of 
each of the connection line, the first connection line, and the 
second connection line is a common bos connection. 

In the parallel computer described above, a structure of 
5 each of the connection line, the fir at connection line, and the 
second connection line is a cross-bus connection. 

In the parallel computer described above, a structure of 
each of the connection line, the first connection line, and the 
second connection line is a star connection . 
10 In the preferred embodiment of the present invention, the 

processing unit in the intermediate stage of the multiprocessor 

Q\ system of a hierarchy structure comprises a processor having 

>i() 

a same function of a normal processor* an instruction memory, 
fjj and a data memory* The processing unit in the intermediate stage 

! " 15 receives a status signal from the lower processing unit, and 
a DMA. controller (as having a data transfer memory of a large 
size compresses received data and decompresses transfer data 
to be transferred , and performs a programmable load dispersion 
or a load dispersion according to the operating state. 

20 

BRIEF DESCRIPTION OF THE DRAWINGS 

These and other objects, features, aspects and advantages 
of the present invention will become more apparent from the 
following detailed description of the present invention when 
25 taken in conjunction with the accompanying drawings, in which; 

FIG.l is a block diagram showing an overview of a 
multiprocessor system having a hierarchy bus structure according 
to a first embodiment as the parallel computer having a hierarchy 
structure of the present invention; 
30 FIG, 2 is a block diagram showing an arrangement of multl 
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chip module 8 on a board on which the parallel computer shown 
In PIG.l is mounted; 

FIG. 3 Is a block diagram showing an overvlav of a 
multiprocessor system having a hierarchy baa structure as the 
parallel computer having a hierarchy structure according to a 
second embodiment o£ the present Invention; 

FIG* 4 Is a block diagram showing a configuration of amultl 
chip module In which the parallel computer Is Implemented; 

FIG- 5 Is a block diagram showing one configuration of an 
Intermediate hierarchy unit; 

FIG* 6 Is a diagram explaining a collision decision among 
objects shown in an image; 

FIG, 7 is a diagram explaining a collision decision among 
objects shown in an Image; 

FIG. 8 Is a diagram explaining a collision decision among 
objects shown In an Image; 

FIG. 9 is a diagram showing the comparison In performance 
between multiprocessor systems of a hierarchy bus structure of 
both a prior art and the present Invention; and 

FIG • 10A and FIG . 10B are diagrams each showing a connection 
structure of the parallel computer having a hierarchy structure 
according to the present Invention. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Other features of this invention will become apparent 
through the following description of preferred embodiments which 
are given for illustration o£ the invention and are not intended 
to be limiting thereof. 

First embodiment 
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FIG.l is a block diagram showing an overview of a 
multiprocessor system Having a hierarchy bus structure according 
to a first embodiment as the parallel computer having a hierarchy 
structure of the present invention. 
5 The multiprocessor system having a hierarchy bus structure 

shown In FIG.l comprises a GHQ main memory of 1 Gbytes, a GHQ 
processor 113, and four SQUAD processing units 120 each of which 
incorporates a plurality of processors (that will be described 
later In detail). Each SQUAD processing unit 120 is Implemented 
10 with a multi-chip module (KCK) . The GHQ processor 113 , the four 
SQUAD processing units 120, and the GHQ main memory 111 axe 
connected through a first level bus 112. 

The six component units, namely, a memory module forming 
the GHQ main memory 111, the GHQ processor 113, and the four 
15 NCKs are connected to each other on a print wiring board 101. 
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i") As shown In FIG.2, each of the four SQARD processing units 120 
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Is mounted as the MCM on the pint wiring board 101. 

FIG. 2 is a block diagram showing an arrangement of HCKs 
on the print wiring board 101 on which the parallel computer 
20 having a hierarchy structure according to the first embodiment 
shown In FIG.l is mounted. In general, the MCM is formed by a 
plurality of unpackaged semiconductor integrated circuits to 
be incorporated In a subsystem In a normal single semiconductor 
chip package. 

25 One type of MCM comprises a substrata (or a board) , a thin 

film connector structure , and a plurality of Integrated circuits 
which are connected to the thin film connector structure and 
surrounded by an epoacy passivation material. The MCM structure 
gives to uses a feature to realise a higher frequency performance 

30 When compared with the print wiring board that is formed by a 
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conventional, plating through-hole and surface mounting 

technology. That la. aa shown In FIG, 2, it la possible to reduce 

both the wiring capacity and a transfer length by packaging the 

nniltiohlps 121 , 123, and the four modules 130 into the multlchlp 
5 module KCM 120. In general , this configuration increases the 

performance of the computer system* 

1ICM requires a high-density structure of wiring In order 

to transfer signals among XC chips 101a to 10 If mounted on a 

common substrate formed by the plural layers 102A to 102E. 
10 By the way, It la possible to use an optional number of the layers 

in order to adopt a dedicated fabrication technology and a wiring 

density to be required In design. 

As shown In FIG. 2. the IC chips 101c and 10 Id correspond 

to the DMA memory module 121 aa one chip and the SQUAD processor 
15 123 , respectively. The IC chips 101a, 101b. 101e # and 10 If 

correspond to the FLIGHT processing units 130, respectively. 

ThB common bus is formed on each of the plural layers 102A to 

102E, 

A multilevel ceramic substrate technology that has been 
20 described in the prior art document Japanese patent laid-open 
publication number JP-A- 10/56036 is used for the configuration 
of the first embodiment shown In FIG. 1- It is, however, possible 
to use another optional technology in a same level. 

In FIG. 2, each of the layers 102A to 102S is formed by 
25 using an insulation ceramic material on which a patterned 
metallzatlon layer has been formed. A part of each of the layers 
102A to 102D has been eliminated, so that amulti- cavity structure 
Is f ozmed. A part of each of the patterned met alls at ion layer 
is exposed around the mult 1- cavity portion. 
30 The exposed part In the layer 102E forms a mounting section 
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for chips. The exposed part Is coated on a metalixation ground 
surface on which the IC chips 10 la to 10 If are mounted by a chip 
mounting technology by using a conductive epoacy, a solder, or 
the like. 

5 Each of the layers 102b to 102D has signal wirings through 

which digital data signals are transferred from the IC chips 
101a to lOlf to MCM Input/output pins or terminals (not shorn) . 

The layer 102A is capable of performing chemical . 
mechanical, and electric protection for lower layers that are 
10 formed in a lower section* In addition to the feature, a package 
cap is also mounted on the layer 102A. 

Printing wirings, I/O pins, and terminals are formed on 
the layers 102B to 102D by using available MCM technology, so 
that the HCM 100 can be connected to outer alroults* In wire 
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15 bonding, bonding pads at one edge of each of the IC chips 101a 

\„\ 

;;:J to 10 If are connected to selected conductors or bonding pads 

u 

in of the layers 10 IB to 10 2D* The configuration described above 

can enlarge the bandwidth of the second level in a lower stage 

y 

when compared with the bandwidth of the printing wiring board 
20 as the upper stage. 

Similarly, a plurality of FIGHTER processing units are 
mounted in the FLIGHT processing unit 130 where the plural FIGHTER 
processing units are connected in a single silicon substrate 
that is higher in signal transfer when compared with the HCM 
25 structure. It is therefore possible to achieve a wider bandwidth. 

Thus, the present invention has a feature to provide that 
the processing units in a lower stage are more Integrated and 
may operate at a higher frequency. 

The GHQ processing unit 110 at the uppermost stage monitors 
30 the entire operation of the parallel computer system. The GHQ 
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processing unit 110 comprises the one chip GHQ processor 113 
and tho GRQ main memory 111 • In the configuration shown In FIG, 1 , 
the number of tbe stages is four, namely, the GHQ processing 
unit 110, tbe SQUAD processing unite 120, the FLIGHT proceeding 
units 130, and tbe FIGHTER processing unite 140. Tbe GHQ 
processing unit 110 is directly connected to tbe four SQUAD 
processing units 120, each comprises the PLIGHT processing unit 
130 and tbe FIGHTER processing unit 140 . Tbe GHQ processing unit 
110, tbe SQUAD processing unit 120, and tbe GHQ main memory 111 
are connected to each other through the first level bus 112 (as 
a common bus) of 32 bit width, and the entire bandwidth is 256 
Mbytes/sec (frequency 66MHz). 

Tbe SQUARD commander processor 123 in each SQUAD processing 
unit 120 controls the entire operation of the unit 120 . The SQUAD 
commander processor 123 is connected to tbe SQUAD instruction 
memory 125, the SQUAD data memory 127, and the SQUAD DMA memory 
121. The SQUAD processing unit 120 is integrated on a single 
semioonduotor chip, as shown in FIG. 2. 

The SQUAD commander processor 123 is directly connected 
to the four FLIGHT processing unite 130 as the following stage. 
Tba four FLIGHT processing unit 130 controls tbe entire operation 
of tbe sixteen FIGHTER processing units 140. 

She SQUAD commander processor 123 is connected to the 
FLIGHT processing unit 130 through the second level bos 114 of 
64 bits/sec. Accordingly, the entire bandwidth becomes 800 
Mbytes/sec (frequency 100MH2). 

The FLIGHT commander processor 133 in each FLIGHT 
processing unit 130 controls the entire operation of each unit 
130 <, The FLIGHT oommaader processor 133 is connected to the FLIGHT 
instruction memory 135, the FL IGHT data memory 137, and the FLIGHT 
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DMA memory 131. The FLIGHT processing unit 130 is Integrated 
on the single semiconductor ahlp of the SQUAD processing unit 
120, as shown In PIG. 2. 

The FLIGHT oonmaader processor 133 Is directly connected 
5 to the sixteen FIGHTER processor 143 In the FIGHTER processing 
unite 140 as the following stage , each of Which Includes a FIGHTER 
memory 141. The FLIGHT commander processor 133 In each FLIGHT 
processing unit 130 la connected to the FIGHTER processor 143 
through the bus 118 of 128 bits/sec. Accordingly, the entire 
10 bandwidth becomes 2128 Mbytes/sec (frequency I33KHz) . The 
operation frequency of the FIGHTER processor 143 Is 533HH*. 
W The GHQ processing unit 110 divides a program (or a task) 

la to a plurality of sub -programs ( or in to a plurality of subtasks ) 
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and sends the divided sub -programs to each of the SQUAD processing 
15 units 120. After the division process, the GHQ processing unit 
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110 compresses the sub-programs (or subtasks) and then the 
compressed them to the SQUAD processing unit 120. There are 
Run- length method or Huffman code method as the compression 
algorithm. The compression method is selected according to the 

20 characteristic of data to be compressed. If it is not necessary 
to use any data compression, the subtasks that have not been 
compressed are transferred to the SQUAD processing units 120. 

In the parallel computer having a hierarchy structure 
according to the present Invention, the task is divided into 

25 a plurality of subtasks, and if necessary, the compression for 
the snbtasks that has been divided are executed, and then 
transferred to the following stage. Therefore the size of the 
subtask is more decreased at the processing unit in a lower stage, 
and the increasing of the bandwidth can be suppressed even if 

30 the operation frequency becomes high, 
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When receiving the task data (or compressed task data if 
necessary) from the GBQ processor 113 In the GEO processing unit 
110, the SQUAD commander processor 123 In the SQUAD processing 
unit 120 sands to the GBQ processing unit 110 the information 
5 that the status of the SQUAD processing unit 120 enters a busy 
state* Then, when the received task has been compressed, the 
SQUAD commander processor 123 decompresses the received task 
data. 

On the contrary, the SQUAD commander processor 123 In the 
10 SQUAD processing unit 120 further divides the received task In 
order to assign the divided task (or the subtask) to each FLIGHT 
CH processing unit 130, After the completion of the division of 

]■:[ the task to obtain the subtaisks, the SQUAD processing unit 120 

I'! j compresses the divided task and then transfers them to the FLIGHT 

15 processing units 130. If It is Improperly or not necessary to 
divide the task, the task that has not been divided is transferred 
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to the FLIGHT processing units 130 . When receiving the task data 
from the SQUAD processing unit 120 (or compressed task data If 
necessary) , the FLIGHT processing unit 130 sends to the SQUAD 

20 processing unit 120 the request to set the status of the FLIGHT 
processing unit 130 to the busy state. Then, when the received 
task has been compressed, the FLIGHT processing unit 130 
decompresses the received task data. 

The FLIGHT processing units 130 further divided the 

25 received task into a plurality of tasks and then transfers the 
divided task data item to each FIGHTER processing unit 140 . Where , 
the task data means the oontent of the processing and necessary 
data. That Is, the main function of both the QUAD processing 
unit 120 and the FLIGHT processing unit 130 as an Intermediate 

30 node Is a scheduling and data transfer* The FIGHTER processing 
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units 140 at the lowermost stage performs the actual processing 
of the task. When receiving the task data, the FIGHTER processing 
units 140 sends to the FLIGHT processing unit 130 In the upper 
stage the request to set the status of the corresponding FIGHTER 
5 processing on It 140 to the busy state, and then the FIGHTER 
processing unit processes the received task data* After the 
completion of the process of the task data , the FIGHTER processing 
unit 140 transfers the operation result to the FLIGHTER commander 
processor 123 In the FLIGHTER processing unit 130, and then the 
10 status of the FIGHTER processing unit 140 Is set to the Idle 
I'jj state. 

8* When detecting the FIGHTER processing unit 140 In the 

Hi) 

h* Idle state, the FLIGHTER processing unit 130 assigns the task 

Q data that has not been processed to the FIGHTER processing unit 

15 140 In the Idle state. When all of the task data Items divided 
by one FLIGHT processing unit 130 have been processed by the 
FIGHTER processing units 140, the FLIGHT processing unit 130 
transfers the operation result to the SQUAD processing unit 120, 
and then this SQUAD processing unit 120 sets the status of the 
20 FLIGHT processing unit 130 from the busy state to the Idle state • 
Like the operation of the FLIGHT processing unit 130 , when 
detecting the FLIGHT processing unit 130 In the Idle state, the 
SQUAD processing unit 120 assigns un-processed task to this 
FLIGHT processing unit 130. Similarly, when receiving the 
25 operation results from all of the FLIGHT processing units 130 
In the lower stage , the SQUAD processing unit 120 sends the 
operation result to the GHQ processing unit In the uppermost 
stage. Thereby, the GHQ processing unit 110 sets the Idle state 
to the SQUAD processing unit 120. 
30 That is, like the operation of the FLIGHT processing unit 
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130 , when detecting the SQUAD processing unit in the idle state 
and Whan there is tin-processed task, the GHQ processing unit 
110 assigns the un- processed task to the SQUAD processing unit 
120. When the SQUAD processing unit 120 completes the operation 
of ail of the tasks from the GHQ processing unit 110 . the operation 
of the given program is completed. 

As described above, the FIGHTER processing units 140 
in the lowermost stage, the FLIGHT processing units 130 and SQ0AD 
processing units 120 in the intermediate stage , and the GHQ 
processing unit 110 in the uppermost stage perform different 
operation to each other. 

That is , because each FIGHTEB processing unit 140 performs 
the actual arithmetic operation . it is not necessary to have 
the function to perform complicated decisions and routines, but 
it is necessary to have a function of a high arithmetic calculation . 
Accordingly, it is preferable that each FIGHTER processor 143 
has plural Integer arithmetic units and floating-point 
arithmetic units. In this embodiment, for example, the FIGHTER 
processing unit 140 includes one integer arltbmetlc unit and 
two floating-point arithmetic units. In this embodiment , hazard 
processing circuits and interrupt circuits to increase high speed 
operation are omitted . Accordingly , when the operation frequency 
is 533 MHz, it is possible for the parallel computer having a 
hierarchy structure of this embodiment to perform the operation 
of 1.066GFLQPS. 

On the other band, the function of the SQUAD processing 
units 120 and the FLIGHT processing units 130 in the intermediate 
stage is a broker, namely, they control the data transfer between 
the upper stage (or the uppermost stage) to the lower stage (or 
the lowermost stage) . Accordingly, it is adequate that each of 
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the SQUAD commander processor 123 and the FLIGHT commander 
processor 133 Incorporate a en arithmetic unit of the smallest 
operation else* In this embodiment, each of the SQUAD commander 
processor 123 and the FLIGHT oommnnrter processor 133 Incorporates 
one Integer arithmetic unit. 

Because the GHQ processing unit 1 10 executes a main program , 
a ge n eral-purpose processor Is used as the GHQ commander 
processor 113, Accordingly, it Is possible to use a 
microprocessor of a high performance for the GHQ commander 
processor 113. 

The configuration of the first embodiment of the present 
Invention la realized based on the following technical Idea. 

The six components , the memory module forming the CTQ 1 
memory 111, the GHQ processor 113 , and the f our multi-chip modules 
120 are synchronization with the clock of 66MHz. in this stage, 
the frequency of this synchronous clock Is suppressed to a 
relatively low value because It Is naoeasary to synchronize the 
six components In a wide area. 

Next, each SQUAD processing unit 120 receives the 
synchronous clock of 66MHz from the GHQ processing unit 110, 
and a Phase Locked Loop (PLL) (not shown) generates the 
synchronous clock of 100MHz that is 1 .5 times of the synchronous 
clock of 66HHz- This synchronous clock of 100MHz Is used as a 
synchronous clock In each SQUAD processing unit 120. The four 
FLIGHT processing units 130, the SQUAD commander processor 123, 
the SQUAD Instruction memory 125, the SQUAD data memory 127, 
and the SQUAD DHA memory 121 operate In synchronization with 
this synchronous clock of 100MHz. one region in the SQUAD 
processing unit 120 is integrated to a part of the area of the 
GHQ processing unit 110 , so that a signal transfer length and 

19 



a signal skew may he decreased, and it Is possible to operate 
at a high frequency. 

Next, eaoh FLIGHT processing unit 130 receives the 
synchronous clock of 100MHz £ron the SQUAD processing unit 120, 
and a PLL ( not shown) or another circuit generates the synchronous 
clock of 133KHz that is approximately 1 . 5 times of the synchronous 
dock of 100MHz. This synchronous clock of 133MBz Is used as 
a synchronous clock In each FLIGHT processing unit 130. The 
sixteen FIGHTER processing units 140 , the FLIGHT commander 
processor 133, the FLIGHT instruction memory 135, the FLIGHT 
data memory 137, and the FLIGHT DMA memory 131 operate in 
synchronisation with this synchronous clock of 133MHz , One region 
in the FLIGHT processing unit 130 is Integrated to a part of 
the area of the SQUAD processing unit 120 , so that it la possible 
to operate at a higher frequency. 

Furthermore, each FIGHTER processing unit 140 receives 
the synchronous clock of 133MHz from the FIGHTER processing unit 
130, and a PLL (not shown) or another circuit generates the 
synchronous clock of 266KBs that is approximately 2 times of 
the synchronous clock of 133NBz . This synchronous clock of 266MHz 
Is used as a synchronous clock in each FIGHTER processing unit 
140. Then, a FLL (not shown) or another circuit: generates the 
synchronous clock of 533HHx that Is approximately 2 times of 
the synchronous clock of 266MHz . This synchronous clock of 533NHz 
is used as an operation clock only for each FLIGHT commander 
processor 133 . The FLIGHT commander processor 133 and the FIGBTR 
memory 141 operate in synchronization with this synchronous clock 
of 266mhz* One region In the FIGHTER processing unit 140 Is 
Integrated to a part of the area of the FLIGHT processing unit 
130, so that it is possible to reduce both a signal transfer 
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length and a signal skew, end also possible to operate at a high 
frequency ■ 

Next, a description gill be given o£ the configuration 
of the intermediate stage, namely, the configuration of the SQUAD 
processing unit 120 and the FLIGHT processing unit 130, la the 
parallel computer according to the present Invention. 

FIG* 5 is a block diagram showing one example of the 
configuration of the Intermediate hierarchy unit such as the 
SQUAD processing unit 130 and the FLIGHT processing unit 130. 

In the configuration of the intermediate stage shown in 
FIG. 5, the general-purpose processor aa the GHQ oomaaudar 
processor 123 Is connected to a Direct Kemory Access (DMA) 
controller 151 of 10 channels. Because this DMA controller 151 
and the general -purpose processor 123 are in a coprocessor 
connect ion. It Is possible to use an available DMA controller. 

The DMA controller 151 Is connected to a bus through which 
a memory 121 of a large memory else (as the SQUAD DMA memory) , 
a connection line to the upper stage, and a connection line to 
the lower stage are connected. A processor core in the 
general -purpose processor 123 has signal lines through which 
a status signal from each processor in the lower stage Is 
transferred. For example, one SQUAD processing unit 120 reoelves 
status signals through four status signal lines connected to 
the four FLIGHT processing units 130 in the lower stage. Each 
status signal line is one bit or more . The status signal Indicates 
whether the processor in the lower stage Is in the busy state 
or the idle state. 

The SQUAD commander processor 123 is connected to the SQUAD 
instruction memory 125 and the SQUAD data memory 127 in which 
programs and data to be used for the SQUAD commander processor 
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123 are stored. These programs expands (or unwinds) data 
transferred from the upper stage If necessary, analyses commands 
also transferred from the upper stage, and performs required 
processes * Then, these programs assign tasks and perform 
5 scheduling, and finally transfer the data to be processed to 
the lower stage. 

As one concrete example, first, the data to be processed 
that are assigned to the target processing unit are transferred 
to the DMA. transfer memory* Second, the data are transferred 
10 to the processing unit In the lower stage that Is capable of 
processing the data. This algorism may be implemented by the 
£j* program that has been stored In the SQUAD data memory 127. In 

"tit 

other words, the processing unit In the Intermediate stage 
fulfills the function as an intelligent DMA system in the entire 
15 parallel computer of the present invention. 

In a case of a system only for a specialized process, 
that is not necessary to apply a versatile purpose, for example 
a graphic simulator and the like, it Is possible to implement 
the processors other than the CSHQ commander processor 113 by 
20 usingnon-Neumann typeDMA processor including the DMA controller 
that is Implemented by the hardware. 

Next , a description will be given of the memory structure 
used In the first embodiment. 

The easiest implementation of the memory structure is that 
25 each of the full processors in the parallel computer has a local 
memory space . Because each processor has a corresponding local 
memory space, it is not necessary to prepare any snoop-bus 
protocol and any coherent transaction. In this case, the memory 
space for the GHQ prooessor 113 Is mapped only In the GHQ main 
30 memory 111. The memory space of the SQUAD commander processor 
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123 is mapped In the SQUAD DNAmamory 12 1 with the SQUAD Instruction 
memory 125 cud the SQUAD data memory 127. 

The memory apace for the GHQ processor 113 and the memory 
space of the SQUAD oommander processor 123 are Independently 
to eaah other. Furthermore , each o£ the different SQUAD commander 
processors 123 is independently to each other. 

Similarly, the memory apace of the FLIGHT oommander 
processor 133 is mapped in tbe FLIGHT DMA memory 131 with the 
FLIGHT instruction memory 135 and the FLIGHT data memory 137. 
The memory space of the FLIGHT oommander processor 133 is 
independently from the memory spaces of both the GHQ processor 
113 and the SQUAD commander processor 133. Moreover, each of 
the FLIGHT commander processors 133 is Independently to each 
other. 

Similarly, the memory space of each FIGHTER processor 143 
is mapped in the corresponding FIGHTER memory 141 of 64Kbytes- 
The memory space of the FIGHTER processor 143 is independently 
from the memory space for the GHQ processor 113 , the memory space 
of each, of the SQUAD commander processor 123. Furthermore, each 
of the FIGHTER processors 143 is independently to each other. 

It is also possible to divide the memory space of the GHQ 
processor 113 into a plurality of memory spaces in order to map 
the divided memory spaces for the full processors in the parallel 
computer according to the first embodiment - In this configuration, 
a move instruction In the GHQ memory 111 is used for the data 
transfer between the upper stage and the lower stage. 

The move instruction for the memory may be Implemented 
as a DMA command to be used between the upper stage and the lower 
stage. In this case, there is a method to share the same address 
by both tbe actual memory of the SQUAD processing unit 1ZO and 
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the actual memory of the GHQ processing unit 110 « However, since 
tba program executed by the GHQ processor 113 controls completely 
the execution state of the full processing units. It is not 
necessary to prepare any snoop-bus protocol and any coherent 
5 transaction. Similarly, the actual memory of the FLIGHT 
processing units 130 and the SQUAD processing units 120 share 
the same address. Zn addition, both the actual memory of the 
FIGHTBB processing units 140 and the actual memory of the FLIGHT 
processing units 130 ahare the same address* 
10 By the way, the multiprocessor system of a hierarchy bus 

structure according to the first embodiment shown in FIG.l has 
the configuration in whiah the four semiconductor chips as the 
four SQUAD HGKs 120, the GHQ processor 113 (not shown In FIG.2) , 
and the main memory 111 (not shown in FIG* 2) are mounted on the 
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On the contrary, the multiprocessor system of a hierarchy 
bus structure according to the second embodiment shown in FIG. 4 
has the configuration in which four SQUAD chips 220, a GHQ 
processor 213 , and a main memory 211 are incorporated in a single 
20 semiconductor chip as a multi-chip module (MCH) . The 
configuration and the operation of the second embodiment will 
be explained later in detail. 

Second embodiment 
25 FIG. 3 is a block diagram showing an overview of a 

multiprocessor system having a hierarchy bus structure as the 

parallel computer having a hierarchy structure according to the 

second embodiment of the present invention. 

The multiprocessor system of a hierarchy bus structure 
30 shown in FIG. 3 comprises a GHQ main memory of 1 Gbytes formed 
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on a single semiconductor chip, a GHQ processor 213 formed on 
e single semiconductor chip, end four SQUAD processing units 
220 eaoh of which Incorporates a plurality of processors (that 
will be described In detail) , Each SQUAD processing unit 220 
is formed on a single semiconductor chip* 

The <3HQ processor 213, the four SQUAD processing units 
220 , and the GEO main memory 211 are connected through a first 
level bus 212* 

She six component units, namely, a memory module forming 
the GHQ main memory 211, the GHQ processor 213, and the four 
SQUAD processing units 220 are mounted on a single multlohlp 
module (MCM) . In general, the MCK la formed by a plurality of 
unpackaged semiconductor integrated circuits that are 
incorporated as sub systems in a package of a normal single 
semiconductor chip* One type of the HOC comprises a substrate 
(or a board) , a thin film connector structure of a desired circuit 
structure, and a plurality of integrated circuits connected to 
the thin film connector structure and surrounded by an epozy 
passivation material. The MCM structure gives to users a feature 
to realize a h igh er frequency performance when compared with 
the print wiring board that is formed by a conventional plating 
through-hole and surface mounting technology. That is, as shown 
In PIG. 4, it is possible to reduce both the wiring capacity and 
a transfer length by packaging the multlohlps on the substrate. 
In general , this configuration increases the performance of the 
computer system* 

FIG. 4 Is a block diagram showing a configuration of the 
mult 1- chip module In which the parallel computer according to 
the second embodiment Is mounted. 

MCM requires a high-density structure of wiring in order 
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to transfer signals among IC chips 201a to 20 If mounted on a 
common substrate formed by the plural layers 202A to 202E. By 
the way, it is possible to use an optional nuaber of the layers 
for the adapting of a dedicated fabrication technology sod a 
wiring density to be required for design. 

As shown in FIG* 4, the ZC chips 201c and 201d correspond 
to the GHQ main memory nodule 211 and the GHQ processor 213, 
respectively. The ZC chips 201a, 201b, 201e r and 20 If correspond 
to the SQUAD processing units 220, respectively* This 
configuration of the second embodiment shown in FIG. 4 is 
different from that of the first embodiment shown in FIG. 2. 

As shown in FIG, 4 . the wiring as a first level bus is formed 
on each of the plural layers 202A to 202E. 

In the configuration of the first embodiment shown in 
FIGs.l and 2, the multilevel ceramic substrate technology that 
has been described In the prior art document Japanese patent 
laid-open publication number JP-A-10/56036 is used. It is, 
possible to use -the same technology fox* the second embodiment • 

In the case of the first embodiment shown in FIG. 2, each 
of the layers 10 2A to 10 2E is formed by using an Insulation ceramic 
material on which a patterned metalizatlon layer has been formed . 

In the configuration of the second embodiment shown in 
FIG . 4 , a part of eadh of the layers 202A to 20 2D has been eliminated , 
so that a mnltl- cavity structure is formed* A part of each of 
the patterned metalizatlon layer in each of the layers 202B to 
202B is exposed around the mnltl -cavity portion. 

The exposed part in the layer* 2025 forms a mounting section 
for chips . The exposed part is coated on a metalizatlon ground 
surface on which the IC chip 201a to 201f are mounted by a chip 
mounting technology sudh as a conductive epoxy, a solder, or 
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the like. 

Each of the layers 202B to 20 2D has signal wirings through 
which digital data signals are transferred from the IC chips 
201a to 20lf to KGH input/output pins or terminals (not shown) . 

the layer 202A Is capable of performing ohemloal , 
mechanical, and electric protection for lower layers that are 
formed in a lower section. In addition to the feature , a package 
cap is also mounted on the layer 102A* 

Printing wirings, I/O pins, and terminals are formed on 
the layers 202B to 202D by using available MCM technology, so 
that the MCM 201 can be connected to outer circuit s* In wire 
bonding, bonding pads at one edge of each of the IC chips 201a 
to 20 If are connected to selected conductors or bonding pads 
of the layers 20 IB to 20 2D* 

The configuration described above can enlarge the 
bandwidth of the first level when compared with the bandwidth 
of the printing wiring board. Similarly, a plurality of FLIGHT 
processing units 230 are mounted in the SQUAD processing unit 
220 where they are connected on a single silicon substrate that 
has an advantage to operate at a high speed, it is thereby possible 
to achieve a wider bandwidth when compared with the MCM structure . 

Thus, the present Invention has a feature to provide that 
the processing units in a lower stage are more Integrated and 
may have a higher operation frequency- 

The GHQ processing unit 210 at the uppermost stage monitors 
the entire operation of the parallel computer system. The GHQ 
processing unit 210 comprises the one chip GHQ processor 213 
and the GHQ main memory 211 . In the configuration shown in PIC. 4 , 
the number of the stages is four, namely, the GHQ processing 
unit 210, the SQUAD processing units 220, the FLIGHT processing 
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unit 230* and the FIGHTER processing units 240. 

The GHQ processing unit 210 is directly connected to the 
four SQUAD processing units 220, the FLIGHT processing units 
230, the FIGHTER processing units 240 as a lover stage. 

The GSQ processing unit 210 and the SQUAD processing units 

220. and the GHQ main memory 211 are connected to each other 
through the ten RAM buses , so that the entire bandwidth becomes 
16Gbytes/seo (frequency 400HBxX2) . 

The six component elements, the memory module forming the 
GHQ main memory 211 and the GHQ processor 213 and the four 
multi-chip modules 220 are synchronised with the synchronous 
clock of 187.5MHz. Accordingly, each SQUAD processing unit 220 
inputs the synchronous clock of 187 • 5MHs from the GHQ processing 
unit 210. 

The SQUARD commander processor 223 in each SQUAD processing 
unit 220 controls the entire operation of the unit 220. The SQUAD 
commander processor 223 is connected to the SQUAD instruction 
memory 225, the SQUAD data memory 227, and the SQUAD DMA memory 

221. She SQUAD processing unit 220 is Integrated on a single 
semiconductor chip, as shown in FIG. 4. 

The SQUAD commander processor 223 is directly connected 
to the four FLIGHT processing units 230 as the following stage. 
The four FLIGHT processing unit 230 controls the entire operation 
of the sixteen FIGHTER processing units 240. 

The SQUAD commander processor 223 Is connected to th * 
FLIGHT processing unit 230 through the bus of 6,144 bit-width. 
Accordingly, the entire bandwidth becomes 388Gbytes/sec 
(frequency 375MHs). 

The four FLIGHT processing unite 230, the SQUAD commander 
processor 223 , the SQUAD instruction memory 225, the SQUAD data 
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memory 227, and the SQUAD DMA memory 221 operate in 
synch root gat Ion with the synchronous dock of 375MH*. 
Accordingly, each FLIGHT processing unit 230inpute the 
synchronous clock of 375KHz from the corresponding SQUAD 
processing unit 220. 

Xhe FLIGHT commander processor 233 In each FLIGHT 
processing unit 230 controls the entire operation of each unit 
230 - The FLIGHT commander processor 233 is connected to the FLIGHT 
instruction memory 2 35 , the FLIGHT data memory 237 , and the FLIGHT 
DMA memory 231* The FLIGHT processing unit 230 Is Integrated 
on the single semiconductor chip o£ the SQUAD processing unit 
220, as shown in FIG. 4. 

The FLIGHT commander processor 233 Is directly connected 
to the sixteen FIGHTER processing units 240 each comprising the 
FIGHTER processing units 243 and the FIGHTER memory of 64kbytes . 

The sixteen FIGHTER processors 243. the FLIGHT commander 
processor 233, the FLIGHT Instruction memory 235, the FLIGHT 
data memory 237, and the FLIGHT DMA memory 231 are synchronized 
by the synchronous clock of 750MHz. Accordingly, each FIGHTER 
processing unit 240 Inputs the synchronous clock of 750MHz from 
the corresponding FLIGHT processing unit 230* 

The FLIGHT processing unit 230 and the FIGHTER processor 
243 are connected to each other through the bus of 1028 bit-width. 
Accordingly, the entire bandwidth becomes 99Gbytes/seo 
(frequency 750MHz). The operation frequency of the FIGHTER 
processor 243 is 1.5GH2* 

The <3&Q processing unit 210 divides a program (or a task) 
into a plurality of subprograms (or a plurality of subtasks) 
and sends the divided sub-programs to each of the SQUAD processing 
units 220 • After the division process of the program or the task. 
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the GEQ processing unit 110 compresses the sub-programs (or 
subtasks) and then transfers the compressed them to the SQUAD 
processing unit 120* There are Ron- length method or Huffman oode 
method as the compression algorithm, the compression method Is 
5 selected according to the characteristic of data to be compressed « 
If It la not necessary to use any data compression, the subtasks 
are transferred to the SQUAD processing units 120 without any 
compression. 

In the parallel computer having a hierarchy structure 
10 according to the present Invention, the task Is divided Into 
a plurality of subtasks, and If necessary , the compression for 
the subtasks that has been divided are executed, and then 
transferred to the following stage. Therefore the size of the 
subtask Is more decreased at the processing unit In a lower stage ; 
15 and the Increasing of the bandwidth can be suppressed even If 
the operation frequency becomes high. 

Hben receiving the task data (or compressed task data If 
necessary) from the GHQ processor 213 In the GflQ processing unit 
210, the SQUAD commander processor 223 In the SQUAD processing 
20 unit 220 sends to the GHQ processing unit 210 the information 
that the status of the SQUAD processing unit 220 enters a busy 
state. Then, when the reoalved task has been compressed, the 
SQUAD commander processor 223 decompresses the received task 
data. 

25 On the contrary, the SQUAD commander processor 223 In the 

SQUAD processing unit 220 further divides the received task data 
In order to assign the divided task to eacah FLIGHT processing 
unit 230. After the division process of the task, the SQUAD 
processing unit 220 compresses the divided task and then 

30 transfers the compressed tasks to the FLIGHT processing units 
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230. If it Is improperly or not necessary to divide the task, 
the task that has not been divided is transferred to the FLIGHT 
processing units 230, When receiving the task from the SQUAD 
processing unit 220 (or compressed task data if necessary) , the 
5 FLIGHT processing unit 230 sends to the SQUAD processing unit 
220 the request to set the status of the FLIGHT processing unit 
230 to the busy state. 

Then, when the received task has been compressed, the 
FLIGHT processing unit 230 decompresses the received task data* 
10 The FLIGHT processing units 230 further divide the received 

;»tl task into a plurality of tasks and then transfers the divided 

task data to each FIGHTER processing unit 240, Where, the task 
!«I data means the content of the processing and necessary data* 

! : 

I'M That is, the main function of both the QUAD processing unit 220 

lAl 

n 15 and the FLIGHT processing unit 230 as an Intermediate node is 
□ a scheduling and data transfer* The FIGHTER processing units 
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240 at the lowermost stage performs the actual processing of 
the task, WHen receiving the task data, the FIGHTER processing 
units 240 sends to the FLIGHT processing unit 230 at the upper 
20 stage the request to set the state of the corresponding FIGHTER 
processing unit 240 to the busy state , and then the FIGHTER 
processing unit 240 processes the received task data. After the 
completion of the process of the task data, the FIGHTER processing 
unit 240 transfers the operation result to the FLIGBTER commander 
25 processor 223 in the FLIGBTER processing unit 230, and then the 
status of the FIGHTER processing unit 240 is set to the idle 
state. 

When detecting the FIGHTER processing unit 240 in the idle 
state, the FLIGHTER processing unit 230 assigns the task data 
30 that has not been processed to this FIGHTER processing unit 240 
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in the idle state. 

When all of the task data Items divided by one FLIGHT 
processing unit 230 have been processed by the FIGHTER processing 
units 240 , the FLIGHT processing unit 230 transfers the operation 
5 result to the SQUAD processing unit 220, and then this SQUAD 
processing unit 220 sets the status of the FLIGHT processing 
unit 230 from the busy state to the Idle state* 

Like the operation of the FLIGHT processing unit 230 , when 
detecting the FLIGHT processing unit 230 In the idle state, the 
10 SQUAD processing unit 220 assigns un-processed task to this 
FLIGHT processing unit 130* 

Similarly, when receiving the operation results from all 
of the FLIGHT processing units 230 at the lower stage, the SQUAD 
processing unit 220 sends the operation result to the GHQ 
15 processing unit 210 in the uppermost stage. Thereby, the GHQ 
processing unit 210 sets the idle state to the SQUAD processing 
unit 220. 

That Is, like the operation of the FLIGHT processing unit 
230, when detecting the SQUAD processing unit 220 In the idle 

20 state and when there is tin-processed task that is not processed, 
the GHQ processing unit 210 assigns the un-prooessed task to 
the SQUAD processing unit 220. When the SQUAD processing unit 
220 completes the operation of all of the tasks from the GHQ 
processing unit 210, the operation of the given program is 

25 completed. 

As described above, the FIGHTER processing units 240 
sb the lowermost stage, the SQUAD processing units 220 and the 
FLIGHT processing units 230 in the intermediate stage, and the 
GHQ processing unit 210 in the uppermost stage performs different 

30 operation to each other. 
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That la , because each FIGHTER processing unit 240 performs 
the actual arithmetic operation, it is not necessary to have 
the function to perform complicated decisions and routines, but 
it is necessary to have a function of a high arithmetic calculation • 
Accordingly, it is preferable that each FIGHTER processor 243 
has plural integer arithmetic units and floating-point 
arithmetic units* In this embodiment, for example, the FIGHTER 
processing unit 240 includes one integer arithmetic unit and 
two floating-point arithmetic units, and hazard processing 
circuits and interrupt circuits to increase operations at high 
speed are emitted* Accordingly, when the operation frequency 
is 1.5GHz, it is possible for the parallel computer having a 
hierarchy structure of this embodiment to perform the operation 
of 24GFLOPS. 

On the other hand, the function of the SQUAD processing 
units 220 and the FLIGHT processing units 230 in the intermediate 
stage is a broker, namely, they control the data transfer between 
the upper stage (or the uppermost stage) to the lower stage (or 
the lowermost stage) . Accordingly, it is adequate that each of 
the SQUAD commander processor 223 and the FLIGHT commander 
processor 233 incorporates an arithmetic unit of tbe smallest 
operation size* In this embodiment, each of the SQUAD commander 
processor 223 and the FLIGHT commander processor 233 incorporates 
one integer arithmetic unit. 

Because the GHQ processing unit 210 executes amain program, 
a general-purpose processor is used as the GHQ commander 
processor 213* It is therefore possible to use a microprocessor 
of a high performance as the GHQ commander processor 213 . 

Accordingly, the configuration of the second embodiment 
of the present Invention is realized based on the following 

S3 



technical idea. 

The six components, the memory module forming the GHQ main 
memory 211, the GHQ processor 213, and the four mult 1-dhip modules 
220 are synchronization with the synchronous clock of 187 . 5HHz . 
In this stage* the frequency of this synchronous clock Is 
suppressed to a relatively low value because It is necessary 
to synchronize the six components placed in a vide area. By the 
way, the OTQ main memory 211 operates based on the clock of 400MHz 
that is used for asynchronous data transfer, not for synchronous 
data transfer. 

Next, each SQUAD processing unit 220 receives the 
synchronous clock of 187*5MHz from the GHQ processing unit 210, 
and a Phase Looked Loop (PLL) (not shown) generates the 
synchronous dock of 37SMHz that is 2 times of the synchronous 
clock of 187.5MHz. This synchronous clock of 375MHz is used as 
a synchronous clock in each SQUAD processing unit 220. The four 
FLIGHT processing units 230, the SQUAD oommander processor 223, 
the SQUAD instruction memory 225, the SQUAD data memory 227 , 
and the SQUAD DMA memory 221 operate in synchronization with 
this synchronous clock of 375MHz. 

One region In the SQUAD processing unit 220 is integrated 
to a part of the area of the GHQ processing unit 210, so that 
it is possible to decrease both a signal transfer length and 
a signal skew, and also possible to operate at a h -»g>* frequency. 

Next, each FLIGHT processing unit 230 receives the 
synchronous clock of 375MHz from the SQUAD processing unit 22o, 
and a PLL ( not shown ) or another circuit generates the synchronous 
clock of 750MHz that is approximately 2 times of the synchronous 
dock of 375MHz. This synchronous clock of 750MHz is used as 
a synchronous clock in each FLIGHT processing unit 230. The 
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sixteen FIGHTER processing units 240. the FLIGHT oamsander 
processor 233, the FLIGHT Instruction memory 235, the FLIGHT 
data memory 237, end the FLIGHT DMA memory 231 operate in 
synchronisation with this synchronous clock of 7S0MB*. 
5 One region In the FLIGHT processing unit 230 is Integrated 

to a part of the area of the SQUAD processing unit 120 # so that 
It Is possible to operate at a higher frequency. 

Furthermore, each FIGHTER processing unit 240 receives 
the synchronous clock of 750MHz from the FIGHTER processing unit 
10 230, and a PLL (not shown) or another circuit generates the 
O synchronous dock of l.SGHs that is approximately 2 times of 

htt 

CIS the synchronous clock of 7 50MHz . This synchronous clock of 1 . 5GRz 
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is used as a synchronous clock in each FIGHTER processing 
unit 240 * Each FLIGHT commander processor 233 and the FIGHTER 

1 5 memory 241 operate In synchronization with this synchronous clock 
of 1.5GHz, The FIGHTER processing unit 240 Is Integrated into 
a small region, so that it is possible to reduce a signal transfer 
length and a signal skew as small as possible, and It Is thereby 
possible to operate at a high frequency. 

20 Although the Internal processes in the FLIGHT processing 

unit 230 operate baaed on the synchronous alock of 755MHz, it 
is difficult for the entire of the GHQ processing unit 210 to 
operate with the clock of 755MH*. Accordingly, the different 
FLIGHT processing units 230 can not operate synchronously* 

25 However, there is no problem If the SQUAD processing units in 
the upper stage may operate synchronously* 

Next, a description will be given of the configuration 
of the intermediate stage , namely, the configuration of the SQUAD 
pxocesslng unit 220 and the FLIGHT processing unit 230, In the 

30 parallel computer of the present invention. 
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As shown in FIG. 5 that baa bean used la the explanation 
of the first embedment, FIG, 5 show the configuration of one 
unit In the Intermediate stage* 

In the configuration of the intermediate stage shown In 
FIG-5, the general - purpose processor as the G8Q commander 
processor 223 Is connected to a Direct Memory Access (DMA) 
controller 151 of 10 channels* Because this DMA controller 151 
and the general -purpose processor 223 are In a coprocessor 
connection. It la possible to use an available DMA controller. 

The DMA controller 151 Is connected to a bus through which 
a memory 221 of a large memory size (as the SQUAD DMA memory) , 
a connection line to the tipper stage, and a connection line to 
the lower stage are connected* A processor core In the processor 
223 has signal lines through which a status signal from each 
processor In the lower stage Is transferred. For example, one 
SQUAD processing unit 220 receives status signals through four 
status signal lines connected to the four FLIGHT processing units 
230 in the lower stage. 

Each status signal line is one bit or more. The status 
signal indicates whether the processor in the lower stage Is 
in the busy state or the idle state. 

The SQUAD commander processor 223 is connected to the SQUAD 
instruction memory 225 and the SQUAD data memory 227 in which 
programs and data to be used for the SQUAD commander processor 
223 are stored. These programs expands (or unwinds) data 
tran sferred from the upper stage if necessary, analyses commands 
also transferred from the upper stage, and performs required 
processes. Then, these programs assign tasks and perform 
scheduling, and f in ally transfer the data to be processed to 
the lower stage. 
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As one concrete example, first, the data, to be processed, 
that are assigned to the target processing unit are transferred 
to the DM1 transfer memory. Second, the data are transferred 
to the processing unit In the lower stage that Is capable of 
processing the data* This algorism may be Implemented by the 
program that has been stored In the SQUAD data memory 227. 

In other words, the processing unit in the Intermediate 
stage fulfills the function as an Intelligent DMA system in the 
entire parallel computer of the present Invention • 

In a case of a system only for a specialised process, that 
is not necessary to apply a versatile purpose, for example a 
graphic simulator and the ll&e, it Is possible to implement the 
processors other than the GHQ commander processor 113 by using 
non-Neumann type DMA processor Including the DMA controller that 
Is Implemented by the hardware* 

Next, a description will be given of the memory structure 
used In the first embodiment. 

The easiest Implementation of the memory structure Is that 
each of the full processors In the parallel computer has a local 
memory space. Because each processor has a corresponding local 
memory spaae. It Is not necessary to prepare any snoop-bus 
protocol and any coherent transaction. In this oase, the memory 
space for the GHQ processor 213 Is mapped only In the GHQ main 
memory 211. The memory space of the SQUAD commander processor 
223 Is mapped In the SQUAD DMA memory 221 with the SQUAD instruction 
memory 225 and the SQUAD data memory 227. 

The memory space for the GHQ processor 213 and the memory 
space of the SQUAD commander processor 223 are Independently 
to each other . Furthermore , each of the different SQUAD commander 
processors 223 Is Independently to each other* 
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Similarly, the memory space of the FLIGHT commander 
processor 233 Is mapped la the FLIGHT DMA memory 231 with the 
FLIGHT instruction memory 235 end the FLIGHT data memory 237. 
The memory space of the FLIGHT commander processor 233 Is 
Independently from the memory spaces of both the GHQ processor 
213 and the SQUAD commander processor 233. Moreover, each of 
the FLIGHT commander processors 233 Is Independently to each 
other. 

Similarly, the memory space of each FIGHTER processor 243 
Is mapped In the corresponding FIGHTER memory 241 of 6 4 Kbyte 8 , 
The memory space of the FIGHTER processor 243 is Independently 
from the memory space for the GHQ processor 213 , the memory space 
of each of the SQUAD commander processor 223. Furthermore, each 
of the FIGHTER processors 243 is independently to each other. 

It is also possible to divide the memory space of the GHQ 
processor 213 into a plurality of memory spaces in order to map 
the divided memory spaces for the f nil processors in the parallel 
computer according to the second embodiment. In this 
configuration, a move instruction In the GHQ memory 211 is used 
for the data transfer between the upper stage and the lower stage . 

The move instruction for the memory may be Implemented 
as a DMA command to be used between the upper stage and the lower 
stage. In this case, there is a method to share the same address 
by both the actual memory of the SQUAD processing unit 220 and 
the actual memory of the GHQ processing unit 210 . However, since 
the program executed by the GHQ processor 213 controls completely 
the execution state of the full processing units. It is not 
necessary to prepare any snoop-bus protocol and any coherent 
transaction. Similarly, the actual memory of the FLIGHT 
processing units 230 and the SQUAD processing units 220 share 
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the same address. In addition, both the actual memory of the 
FIGHTER processing unite 240 and the actual memory of the FLIGHT 
processing units 230 share the same address* 

By the way, although the preferred first and second 
5 embodiments described above have shown the multiprocessor system 
of a hierarchy bus structure, as shown In FIGs • 1 to 4 , the present 
invention Is not limited by these configurations. 

PIG . IDA and FIG . 10B are diagrams each Showing a connection 
structure of processing units In the parallel computer having 
10 a hierarchy structure according to the present invention* For 
example, as shown in both the FIG. 10A and FIG. 10B r It Is possible 
to apply the concept of the present Invention to various 
connection configurations, namely, it is possible to form the 



Mi connection among the FLIGHT processing unit 130 and the 

15 corresponding FIGHTER processing units 140 In the parallel 



computer of the present Invention based on a cross-bus connection 
(shown In FIG.10A), a star connection (shown in FIG.10B), or 
other connections * 

20 <Dlf ferenoe In performance between the present Invention and 
prior art> 

Next, a description will be given of the explanation of 
the difference in performance between the multiprocessor system 
having a hierarchy bus structure of the present invention and 
25 that of the prior art. 

First, in the multiprocessor system of a hierarchy bus 
structure of the second embodiment shown in FIG. 3 in which each 
stage has only a cache , the estimation of the data transfer amount 
In a collision decision application is as follows : 
30 In the consideration to perform the collision decision 
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between objects shown In image, each object Is divided into 
regions, each Is called to as a bounding shape. Bach collision 
decision la performed for all of the combination of the bounding 
shapes. When the bounding shape Is a sphere shape, the collision 
5 decision between one bounding shape and another bounding shape 
can be expressed by following calculations 

(x x - x 2 ) 2 +(y r Ya) 2 +(Zi- z*> 2 

< (r x - r 2 )\ 

The content of amount of the calculation Is as follows: 
10 (1) Load for eight elements x 1# y 1# z x , T x , x 2 , y 2 # Z * . 

and r a X 4 bytes »32 bytes; 

(2) Six addition/subtractions; 

(3) Four multiplications; and 

(4) One comparison. 

1 5 Accordingly , the total requires the calculation of 8 loads 

and 11FP- 

Zn the system including the FIGHTER processor at the 
lowermost stage having the calculation ability of 

2FLsX 1.5GHz = 3GFLOPS, 
20 each FIGHTER processor has a collision decision ability 

of 3GFLOPS/11FP =» 275 MHz times/sec* 

This FIGHTER processor consumes data of 

3GFLOPS/llFFX32bytes « 8.75 Gbytes/seo. 
When the system has 128 processors X 2FP X 1 „ 5GHz»304GFIiOFS , 
2B the collision decision ability becomes 3B4GFLOPS/11FP 

= 34.9G times/sec. 
Xn l/60second. It becomes 384GFLOPS/11FP/60 

= 580M times/frame. 
This equals to V" (2X580M) » 34. 134MHz and means the ability 
SO to perform the collision deoision between bounding shapes over 
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30,000perl/60seo« The bandwidth to be necessary for tills ability 
Is: 

FLIGHT bus : 8.75Gbytres/sec X a - 70Gbytes/seo; and 
SQUAD bus : 70Gbytes/seo X 4 = 280Gbytes/seo« 
5 Next, a description will be given of the cose of the data 

expansion and the uniform load dispersion by using the processor 
In the intermediate node* 

As shown in FIG. 6, for example, the GHQ processing unit 
in the uppermost stage divides tasks of the source side and the 
10 target side into subgroups (for example, 10 subgroups) , and than 



CI 

the processors process the divided subgroups per mXm. The 
subgroups are dispersed by the DMA to the processors In the idle 
state. That is, the tasks are processes in the load dispersion 
while checking which processor is in the idle state. Thereby, 
1 5 even if one processor detects the collision so that the processing 
time becomes long, the entire processors can disperse the tasks . 

For example, when it is necessary to perform the collision 
decisions for 100 , 000 bounding shapes, the 1/4 data for the total 
collision decisions are dispersed to each of the four SQUAD 
20 commander processors* 

As one example, as shown in FIG- 7, the SQUAD 1 as the SQUAD 
commander processor handles 1 to n/2 bounding shapes, the SQUAD 
2 as the SQUAD commander processor handles 1 to n/4 and (n/2)+l 
to n bounding shapes , the SQUAD 3 as the SQUAD commander processor 
25 handles the (n/4)+l to n/2 and (n/2}+l to n bounding shapes, 
and the SQUAD 4 as the SQUAD commander processor handles (n/2) +1 
to n bounding shapes . The GHQ processor 213 in the GHQ processing 
unit 110 performs these load dispersion and the DMA transfer. 
Of course, as shown in FIG. 8, it is equivalent that the 
30 SQUAD 1 handles 1 to n/2 bounding shapes, the SQUAD 2 handles 
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(n/2)+l to3n/4andlto (n/ 2) bounding shape*, the SQUAD 3 handles 
the (3n/4)+l to n and 1 to n/2 bounding shapes, and the SQUAD 
4 hand lea (n/2)+l to n bounding shapes. 

Next , each SQUAD processing unit in the Intermediate stage 
disperses the tasks by the same manner described above. 

The SQUAD commander processor handles the load dispersion 
and the DMA transfer. For example # as shown in the configurations 
of FIO.l and PIG. 3, the SQUAD 2 (as the SQUAD processing unit) 
disperses the load Into the FLIGHT 1 to FLIGHT 4 (as the FLIGHT 
processing units). In this case, there is no difference of the 
dispersion efficiency when compared with that of the GHQ 
processing unit . Because there are the sixteen FLIGHT processing 
units in the system, 1/16 collision decisions in the total 
collision decisions to be processes are assigned to each FLIGHT 
processing unit when the total number of the collision decisions 
are equally divided by 16 . The maximum data amount tos be stored 
in each FLIGHT processing unit Is approximately (1/4+1/9*) 3/8 
of the total amount: of data. 

Each FLIGHT commander processor further divides the 
received data into small- sized regions based on the sub-group 
method described above* The amount of the division is determined 
based on the received data amount. 

The group of the divided collision decisions is assigned 
to each FLIGHT commander processor in order to execute the 
collision decision operation. Saoh FLIGHT commander processor 
executes a flat decision for the collision. 

The amount of this data transfer will be estimated in an 
optimized case, when the GHQ bus transfers 1.6 Mbytes data to 
the four SQUAD processing units and updates the data per 1/60 
seconds, the speed of the data transfer becomes * 
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1.6MbytesX4 (SQU^D processing units) -r 1/60 seconds - 
384 Mbytes /sec. 

Because SQUAD bos transfers approximately SSOKbytes 
(578904 bytes} to tbe four FLIGHT processing units, the required 
data bus batidwl rttti becomes: 

580 Kbytes X 4 (FLIGHT processing units) -5- 1/60 seconds 
- 139.2 Mbytes /sec. 

On tbe otber band, tbe data bandwidth required for tbe 
FLIGHT bus becomes: 

1110/(l/60seconds)Xi6Kbytes - approximately lGbytea/seo. 
Tbls value is approximately 1/140 of the 140 Gbytes/seo of tbe 
prior art. FIG. 9 Is a diagram allowing a comparison In operation 
between multiprocessor systems of a bleraroby bus structure of 
botb a prior art and tbe present Invention. That Is, FIG* 9 shows 
tbe estimation results described above. 

Accordingly , tbe bleraroby of tbe unit s In tbe 
multiprocessor system of a hierarchy bus structure of the present 
Invention can suppress a clock skew and execute high speed 
processors of daslxed numbers In front end technology In 
parallel. 

While tbe above provides a full and complete disclosure 
of the preferred embodiments 1 and 2 for the parallel computer 
having a bleraroby structure according to the present Invention , 
various modifications, alternate constructions and equivalents 
may be employed without departing from the scope of the Invention 
by a person with ordinary skill in the art to which tbe present 
invention pertains. Therefore tbe above description and 
illustration should not be construed as limiting the scope of 
tbe invention, which is defined by the appended claims. 
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